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Abstract 

A recent result of Moshkovitz [Mosl4] presented an ingenious method to provide a com¬ 
pletely elementary proof of the Parallel Repetition Theorem for certain projection games via a 
construction called fortification. However, the construction used in [Mosl4] to fortify arbitrary 
label cover instances using an arbitrary extractor is insufficient to prove parallel repetition. In 
this paper, we provide a fix by using a stronger graph that we call fortifiers. Fortifiers are graphs 
that have both £i and £2 guarantees on induced distributions from large subsets. 

We then show that an expander with sufficient spectral gap, or a bi-regular extractor with 
stronger parameters (the latter is also the construction used in an independent update [Mosl5] 
of [Mosl4] with an alternate argument), is a good fortifier. We also show that using a fortifier 
(in particular £2 guarantees) is necessary for obtaining the robustness required for fortification. 


1 Introduction 

Label-cover and general two-prover games 

A label cover instance is specified by a bipartite graph G = ((X, Y), E), a pair of alphabets Ex and 
Ey and a set of constraints tpe ■ Ex —t Ey on each edge e G E. The goal is to label the vertices of X 
and Y using labels from Ex and Ey so as to satisfy as many constraints are possible. 

This problem is often viewed as a two-prover game. The verifier picks an edge (x, y) at random 
and sends x to the first prover and y to the second proven They are to return a label of the 
vertex that they received, and the verifier accepts if the labels they returned are consistent with 
the constraint The value of this game G, denoted by val(G), is given by the acceptance 

probability of the verifier maximized over all possible strategies of the provers. These are also 
called projection games as the constraints are functions from Ex to Ey. They are called general games 
if the constraint on each edge is an arbitrary relation ^ Ex x Ey. 

These two notions are equivalent in the sense that val(G) is exactly equal to the maximum 
fraction of constraints that can be satisfied by any labelling. 
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This problem is central to the PCP Theorem [AS98, ALM+98] and almost all inapproximability 
results that stem from it. The (Strong) PCP Theorem can be rephrased as stating that for every 
£ > 0, it is N P-hard to distinguish whether a given label cover instance has val(G) = 1 or val(G) < 
£. An important step is a way to transform instances with val(G) < 1 — e to instances G' with 
val(G') < e. This is usually achieved via the Parallel Repetition Theorem. 


Parallel Repetition 

The /c-fold repetition of a game G, denoted by G^, is the following natural definition — the verifier 
picks k edges ,{xk,yk) from E uniformly and independently, sends {xi,...,Xk) and 

{yi,...,yk) to the provers respectively, and accepts if the labels returned by them are consistent 
on each of the k edges. 

If val(G) = 1 to start with then val(G^) still remains 1. How does val(G*^) decay with k if 
val(G) < 1? Turns out even this simple operation of repeating a game in parallel has a counter¬ 
intuitive effect on the value of the game. It is easy to see that val(G^) > val(G)^ as provers can use 
a same strategy as in G to answer each query (x,,i/,). The first surprise is val(G*^) is not val(G)^, 
but sometimes can be much larger than val(G)*^. Fortnow [For89] presented a game G for which 
val(G^) > val(G)^, Feige [Fei91] improved this by giving an example of game G with val(G) < 1 
but val(G^) = val(G). Indeed, there are known examples [Razll] of projection games where 
val(G) = (1 — e) but val(G*^) > (^1 — eVk^ for a large range of k. 

The first non trivial upper bound on val(G^) was proven by Verbitsky [Ver96] who showed 
that if val(G) < 1 then the value val(G^) must go to zero as k goes to infinity. It is indeed true 
that val(G^) decays exponentially with k (if val(G) < 1). This breakthrough was first proved by 
Raz [Raz98], and has subsequently seen various simplifications and improvements in parameters 
[Hol09, Raoll, DS14, BG14]. The following statements are due to Holenstein [Hol09], Dinur and 
Steurer [DS14] respectively. 

Theorem 1.1 (Parallel repetition theorem for general games). Suppose G is a two-prover game such 
that val(G) < 1 — £ and let |Ex| |Ey| < s. Then, for any k> 0, 

val(G'^) < 


Theorem 1.2 (Parallel repetition theorem for projection games). Suppose G is a projection game such 
that val(G) < p. Then, for any k >0, 


val{G^) < 



Although a lot of these results are substantial simplifications of earlier proofs, they continue to 
be involved and delicate. Arguably, one might still hesitate to call them elementary proofs. 


Recently, Moshkovitz [Mosl4] came up with an ingenious method to prove a parallel repetition 
theorem for certain projection games by slightly modifying the underlying game via a process that 
the author called fortification. The method of fortification suggested in [Mosl4] was a rather mild 
change to the underlying game and proving parallel repetition for such fortified projection games 
was sufficient for most applications. The advantage of fortification was that parallel repetition 
theorem for fortified games had a simple, elementary and elegant proof as seen in [Mosl4]. 
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1.1 Fortified games 

Fortified games will be described more formally in Section 2, but we give a very rough overview 
here. Moshkovitz showed that there is an easy way to bound the value of repeated game if we 
knew that the game was robust on large rectangles. We shall first need the notion of symmetrized 
projection games. 

Symmetrized Projection games. Given a projection game G on ((X, Y), E), the symmetrized 
game Ggym is a game on ((X, X),E') such that for every y EY with {x,y), {x',y) G E, there is an 
edge {x,x') G E' with the constraint Ti(^^^y){crx) = 7Z(^x',y){^x’)- 

For projection games, it would be more convenient to work with the above symmetrized ver¬ 
sion for reasons that shall be explained shortly. It is not hard to see that val(G) and val(Gsym) are 
within a quadratic factor of each other. Thus for projection games, we shall work with the game 
Gsym instead of the original game G. 

Definition 1.3 ((^, e)-robust games). Let G be a two-prover game on ((X, Y),£). For any pair of sets 
S C X, T C Y,let GsxT be the game where the verifier chooses his random query {x,y) G E conditioned 
on the event that a: G S and y eT. 

G is said to be (3, 6)-robust if for every S,T X with |S| > ^|X| and |r| > (i| Y| we have that 

val(GsxT) < val(G)-Fe. 

Theorem 1.4 (Parallel repetition for robust projection games [Mosl4]). Let Gbe a projection game on 
abi-regular bipartite graph ((X,Y),E) with alphabets Xx andXy. For any positive integer k, if e, 3 > 0 
are parameters such that 23 \'Ly\^^^ < £ and Gsym is {3,e)-robust, then^ 

valjG’jy^) < (val{Gsym) + s)’'+ ke. 

Not all projection games are robust on large rectangles, but Moshkovitz suggested a neat way 
of slightly modifying a projection game and making it robust. This process was called fortification. 

On a high level, for any two-prover game, the verifier chooses to verify a constraint corre¬ 
sponding to an edge (x,y) but is instead going to sample several other dummy vertices and give 
the provers two sets of D vertices {xi,...,xd} and {yi, ...,y]j} such that x = x, and y = yy for 
some i and j. The provers are expected to return labels of all D vertices sent to them but the verifier 
checks consistency on just the edge (x,y). This is very similar to the "confuse/match" perspective 
of Feige and Kilian [FK94]. 

To derandomize this construction, Moshkovitz [Mosl4] uses a pseudo-random bipartite graph 
where given a vertex w, the provers are expected to return labels of all its neighbours (Definition 2.1) 
The most natural candidate of such a pseudo-random graph is an (3, £)-extractor, as we really want 
to ensure that conditioned on "large enough events" S and T, the underlying distribution on the 
constraints does not change much. This makes a lot of intuitive sense, since on choosing a random 
element of S and then a random neighbour, the extractor property guarantee that the induced 
distribution on vertices in X is e-close to uniform. Thus, it is natural to expect that conditioning 
on the events S and T should not change the underlying distribution on the constraints by more 

^The following is the corrected statement from [Mosl5]. 
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than 0(e). This was the rough argument in [Mosl4], which unfortunately turns out to be false. 
We elaborate on this in Section 3.2 and Appendix A. 

A recent updated version [MoslS] of [Mosl4] provides an different argument for the fortifica¬ 
tion lemma using a stronger extractor. We discuss this at the end of Section 1.2. 


1.2 Our contributions 


We present a fix to the approach of [Mosl4], by describing a way to transform any given game 
instance G into a robust instance G* with the same value following the framework of [Mosl4] but 
using a different graph for concatenation, and a different analysis. 

We first describe a concrete counter-example to the original argument of [Mosl4] in Section 3.2, 
that shows concatenating (Definition 2.1) with an arbitrary (^i, e)-extractor is insufficient. In fact, 
as we show in Appendix B, concatenating with any left-regular graph with left-degree by o(l/e<i) 
fails to make arbitrary instances (<5, £)-robust. We instead use bipartite graphs called/orf/^’ers, 
defined below. 

Definition 1.5 (Fortifiers). A bipartite graph H = ((W, X),Eh) is an (^,£i,£ 2 )-fortifier if for any set 
sew such that |S| > <5|W|, if n is the probability distribution on X induced by picking a uniformly 
random element wfrom S, and a uniformly random neighbor x ofw, then 


|7r — u|i 

l|2 

TT — u 


< 

< 


£n 


£2 

ixr 


Notice that a fortifier is an extractor, with the additional condition that the £ 2 -distance of re 
from the uniform distribution is small. This is what enables us to show that concatenation with a 
fortifier produces a robust instance. 

Theorem 1.6 (Fortifiers imply robustness). Suppose G is a general two-prover game on a bi-regular 
graph ((X,y),£). Then, for any e, S > 0, if Hi = ((W,X),Ei) and H 2 — ((Z,y),E 2 ) are 
fortifiers, then the concatenated game G* = Hi o G o H 2 is (d, 0{e))-robust. 

In particular, bipartite spectral expanders are good fortifiers, as Lemma 2.8 shows. This gives 
us our main result which follows from Lemma 2.8 and Theorem 1.6: 


Corollary 1.7. Let G be a general two-prover game on a bi-regular graph ((X,Y),E). For any e, 3 > 0, 
if Hi = ((W,X),Ei) and H 2 = ((Z,y),E2) are two A-expanders (Definition 2.3) with A < e^/3 then 
concatenated game G* = Hi o G o H 2 is {3, Ae)-robust. 

As one would expect, the condition on the fortifier can be relaxed if the underlying graph of the 
original label cover instance is a spectral-expander. We prove the following theorem. Theorem 1.6 
follows from this theorem by setting Aq = 1. 

Theorem 1.8. Let G be a two-prover game on bi-regular graph ((X,y),E) where G is an Ao-expander. 
Then for any e,3 > 0, if Hi = ((W,X),Ei) and H 2 = ((Z,y),E2) are {3, e, {e/ Aq)) -fortifiers, then the 
concatenated game G* = Hi o G o H2 is {3,0{e))-robust. 

One could ask if the definition of a fortifier is too strong, or if a weaker object would suffice. 
We argue in Section 3.1 that if we proceed through concatenation, fortifiers are indeed necessary 
to make a game robust. 
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Bipartite Ramanujan graphs of degree ©{1/e^d) have A < C'/S and are therefore good forti¬ 
fiers. In Appendix B, we show that this is almost optimal by proving a lower bound of n(l/eci) on 
the left-degree of any graph that can achieve (^,£)-robustness. This shows that our construction 
of using expanders to achieve robustness is almost optimal, in terms of the degree of the fortifier 
graph. Note that the degree of the fortifier is important as the alphabet size of the concatenated 
game is the alphabet size of the original game raised to the degree. There are known explicit con¬ 
structions of bi-regular e)-extractors with left-degree poly(l/£)polylog(l/(i). But the lower 
bound in Section 3.1 shows that (^,e)-extractors are not fortifiers if ^ <C e, which is usually the 
relevant setting (see Theorem 1.4 and Lemma 1.9). 

Though all the above results are stated for bi-regular games, any two-prover game can be easily 
converted to one on a bi-regular graph or roughly the same value via standard tricks. We outline 
such a construction (similar to the construction in [DH13] for projection games) in Appendix D. 

Independently, the author of [Mosl4] came up with a different argument to obtain robustness 
of projection games by using a (ci,e^)-extractor. This is described in an updated version [MoslS] 
present on the author's homepage. 

It is also seen from Theorem 1.8 that bi-regular {S, e^)-extractors are indeed {S, e, £)-fortifiers as 
well. Using an expander instead is arguably simpler, and is almost optimal. 

Remark. Although this fix provides a proof of a Parallel Repetition Theorem for projection games following 
the framework of[Mosl4], the degree of the fortifier is too large to get the required P CP for proving optimal 
hardness of the Set-Cover problem that Dinur and Steurer [DS14] obtained. See [MoslS] for a discussion 
on this. 

Remark about parallel repetition for general games 

A fairly straightforward generalization Theorem 1.4 to robust general games on bi-regular graphs 
is the following. 

Lemma 1.9 (Parallel repetition for general robust games). Let Gbea general two-prover game on a bi¬ 
regular graph ((X, Y),E) with alphabets Ex and Ey. For any positive integer k, ife, S > 0 are parameters 
such that 2^|Ex x Ey|^“^ < e and G is {5,e)-rohust, then 

val(G^) < (val(G)-|-e)^-|-fce. 

But it is to be noted that the fortification procedure via concatenating a fortifier makes |Ex| = 
exp(l/^) and in such scenarios <5|Ex| S> 1 making it infeasible to ensure 2(51 Ex x Ey|^“^ < £. 
Hence, though Lemma 1.9 may be useful in cases where we know that the game G is robust via 
other means, the technique of fortification via concatenation increases the alphabet size too much 
for Lemma 1.9 to be applicable. 

For the case of projection games, this is not an issue as Theorem 1.4 only requires 2(5|Ey|^^^ < e 
and concatenating Ggym by a fortifier only increases |Ex| and keeps Ey unchanged. Thus, one can 
indeed choose £ and 5 small enough to give a parallel repetition theorem for a robust version of 
an arbitrary projection game. 
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Figure 1: Concatenated Games 


2 Preliminaries 

Notation 

• For any vector a, let |a|^ := \a.i\, and ||a|| := ^ YLi be the and £ 2 -norms respectively. 

• We shall use us to refer to the uniform distribution on a set S. Normally, the set S would be 
clear from context and in such case we shall drop the subscript S. 

• For any vector a, we shall use all to refer to the component along the direction of u, and a^ 
to refer to the component orthogonal to u. 

• We shall assume that the underlying graph for the games is bi-regular. This is more or less 
without loss of generality via standard sampling tricks (see Appendix D). 

We define the concatenation operation of a two-prover games with a bipartite graph that was 
alluded to in Section 1.1. 

Definition 2.1 (Concatenation). Given a two-prover game on a graph G — ((X,Y),E) with a set of 
constraints xp, a pair of alphabets Ex and Ey, bipartite graphs Hi = ((W, X), Ei) with left degree Di, and 
H 2 = ((Z, y), E 2 ) with left-degree D2, the concatenated game is a game on the (multi) graph Hi o G o 
H 2 = ((W, Z),£hioGoH2) Ew = E^i and Ez = Ey^. Eabel of a vertex w ^ W (z ^ Z) can be 
thought of labels to its neighbors in Hi(H 2 ) in a fixed order. For any edge {w,z) G EhioGoHj/ Ihere exists 
{x,y) G E such that {w,x) G Ei, and {z,y) G E2. The constraint for this edge first obtains the label of x 
from w, and similarly obtains the label for y from the label ofz, and checks the constraint t/’(x,i/) according 
to the game G. 

Remark. As mentioned earlier for projection games, as in [Mosl4] we shall work with symmetrized version 
Gsym- In Gsym which is played on ((X,X),Esym), concatenating both sides with the same Hi ensures that 
the resulting game G* is still a symmetrized projection game, and that the concatenation operation only 
changes Ex and leaves Ey unchanged for the underlying projection game. 

We state the results in a general setting as the focus here would be mainly on the study of distributions 
of edges of sub-graphs of concatenated graphs. 
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Lemma 2.2 (Concatenation preserves value). [Mosl4] Given any two-prover game on a bi-regular 
graph G, if Hi and H 2 are bi-regular graphs, then we have: 

val(Hi o G o H 2 ) = val(G). 


Expanders, extractors and fortifiers 

Definition 2.3 (Expanders). For any bi-regular bipartite graph H = ((X,Y),E) with |X| = |Y| and 
(left) degree D, we shall use A(H) to denote 


A(H) = max —n—n“ 

v±u V 


where the matrix H is an |y| x |X| matrix (rows indexed by vertices in Y, and columns by vertices in X) 
defined by H{y,x) = 1/D if {x,y) G E and it is 0 otherwise. For any A > 0, a bi-regular bipartite graph 
H is an A-expander if A(H) < A. 

More generally^, if\X\ f \Y\, we define A(H) as follows: 


A(H) 


def 


max 

v±u 




Informally, A(H) measures "how much more does the matrix H shrink v ± ux compared to ux”? 

Lemma 2.4 (Explicit expanders [BL06]). For every D > 0, there exists a fully explicit family of bipartite 
graphs {Gi}, such that G/ is D-regular on both sides and A(G;) < D^^^^(log 

Definition 2.5 (Extractors). A bipartite graph H = ((X, Y),E) is an {S,e)-extractor if for every subset 
sex such that |S| > ^|X|, ifn is the induced probability distribution on Y by taking a random element 
of S and a random neighbour, then 

Itt —u|;^ < e. 

Lemma 2.6 (Explicit Extractors [RVWOO]). There exists explicit [5, e)-extractors G = (X,Y,E) such 
that |X| = 0(|Y|/^) and each vertex ofX has degree D = 0(exp(poly(loglog(l/(i))) ■ (1/e^)). 

Our earlier definition of a fortifier (Definition 1.5) has properties of both an expander and an 
extractor. Indeed, we can build fortifiers by just taking a product an expander and an extractor. 

Lemma 2.7. If Hi = {{V,W),Ei) is a bi-regular {3, e)-extractor, and if H 2 = ((W,X),E 2 ) is a bi-regular 
A-expander, then the product graph Hi ■ H 2 is an [3, e, A^e/3)-fortifier. 


Proof. Let H 2 be the normalized adjacency matrix of graph H 2 and let 71$ denote the probabil¬ 
ity distribution on W obtained by picking an element of S C y uniformly and then choosing a 
random neighbour in Hi. Thus, H 2 TI$ is the probability distribution on X induced by the uni¬ 
form distribution on S and a random neighbour in Hi ■ H 2 . We want to show for all S such that 
|S|>^|V|, 


|H 2 7rs — u|i < e and 


\\H27zs - up < 


Ah/3 

W 


^We are not sure if this definition is standard, but is 
proof. 


a natural generalization and precisely what we need in our 
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The first inequality is obtained as | H 2 775 — u 1 1 = | H 2 ( tts — u) 1 1 < | Tig — u 1 1 <e, where we use the 
fact that \H 2 v\-^ < for any v and any normalized adjacency matrix, and j/Ts ~ u|i < e follows 
form the extractor property of Hi. 

As for the second inequality, observe that 

II^Ts — u|| < max( 7 rs(w)) ■ j/rs — u|i < e ■ max( 7 Ts{w)). 
w€W w€W 


For a bi-regular extractor^ Hi of left-degree D, the degree of any w G Wis (ll/l ■ D/\W\) and the 
number of edges out of S is least <51^1 • D. Hence, maxi„ ns{w) < 1/(<5|W|), which is achieved if 
all neighbours of w are in S. Therefore, 


I l|2 ^ (£/^) 

2IWI 


||H2(7rs -u)f < A- 


|X| 




|X| |W| 


A2. {e/3) 


□ 


In particular, any bi-regular (ci, ej-extractor is a (ci, e, e/^)-fortifier. Hence, if the underlying 
graph G of the two-prover game is a '/3-expander, then Theorem 1.8 states that merely using an 
{3, £)-extractor as suggested in [Mosl4] would be sufficient to make it {3, 0(e))-robust. 

Also, since any graph is trivially a 1-expander, a bi-regular (^,£^)-extractor is also an {3,e,e)- 
fortifier. The following lemma also shows that expanders are also fortifiers with reasonable pa¬ 
rameters as well. 


Lemma2.8. Let H = {{W,X),Eh) be any A-expander. Then, for every 3 > 0, H is also a {3, '/A?-/3,3?-/3)- 
fortifier. 

In particular, if A < £'/3, then H is an (3, e, e)-fortifier. 

Proof. Let H be the normalized adjacency matrix of H. Let S C W such that |S| > ^|1V|. We have. 


u 


S II ^ 


1 

3\W\' 


Hence, by the expansion property of H, 


||Hus - u||^ 


\Hujf<A^ 


M 

|x| 


iu^r< 


A^/3 


|Hus — u|i < VA ^/3 follows from above and Cauchy-Schwarz inequality. 


□ 


Although Lemma 2.8 shows that expanders are also fortifiers for reasonable parameters, the 
construction in Lemma 2.7 is more useful when the underlying graph for the two-prover game 
is already a good expander. For example, if the underlying graph G was a ^i-expander, then 
Theorem 1.8 suggests that we only require a (^,e,£/^)-fortifier. Lemma 2.7 implies that an (^i,e)- 
extractor is already a (3, e, e /^)-fortifier and hence is sufficient to make the game robust. The main 
advantage of this is the degree of <i-expanders must be 0(1/^^) whereas we have explicit {3,e)- 
extractors of degree (1/e^) exp(polyloglog(l/^)) which has a much better dependence in <5. This 
dependence on 3 is crucial for certain applications. 

^The bound on the right-degree guaranteed by bi-regularity is crucial for this claim. Without this, extractors are not 
sufficient for fortification (Section 3.2). 
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3 Sub-games on large rectangles 

Consider a concatenated general game G* = Hi o G o H 2 on {{W,Z),Eh^oGoH 2 ) S C W and 
T Q Z. Let jis (or f/j) denote the induced distributions on X(or Y) obtained by picking a uniformly 
random element of S (or T) and taking a uniformly random neighbour in Hi (or H 2 ). That is, the 
degree of any x G X (or y G Y) within the set S (or T) is proportional to ys(^) (or }iT{y)) (See 
Figure 2). 



Figure 2: Sub-games on large rectangles 


In a subgame (G*)sxT/ the distribution on verifier checking the underlying constraint on (x,y) 
is given by the following expression: 


nx,y 


}is{x)}iT{y) 

E M^)VT{y)' 

{x,y)eG 


(3.1) 


One way to show that the concatenated game G* is {3, 0(e))-robust would be to show that the 
above distribution 7Zx,y is 0(e)-close to uniform whenever |S|, |T| have density at least 3 because 
then the distribution on constraints that the verifier is going to check in Gg^ j is 0(e) close to the 
distribution on constraints in G. Hence, up to additive factor of 0(e) the quantity val(G*)sxT is 
same as val(G). The main question here what properties should Hi and H 2 satisfy so that the 
above distribution is close to uniform? 


3.1 Fortifiers are necessary 

To prove that fortifiers are necessary, we shall restrict ourselves to games on graphs G = ((X, X), E). 
In such a setting, we can choose to concatenate with the same graph H both sides. We show that if 
a bipartite graph H = ((W, X), Eh), makes a game on a particular graph G, {3, 0(e))-robust, then 
H is a good fortifier. 

As mentioned earlier, if the graph G had some expansion properties, then the requirements on 
the graph H to concatenate with can be relaxed. Thus, naturally, the worst case graph G is one 
that expands the least — a matching. 

Lemma 3.1 (Fortifiers are necessary). Let e,3 > 0 be small constants. Let H = {{W,X),Eh) be a 
bi-regular graph, and let G = ((X, X), E) be a matching. Suppose that for every subset S, T C W with 
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|S|, |r| > ^|W|, the distribution (defined in Equation (3.1)) induced by the game {H o G o H)sxT on the 
edges ofG is e-close to uniform. Then, for every S C W with |S| > W|, 


IFs-u|i = e, (3.2) 

Whs-uf = (3.3) 

Proof. It is clear that (3.2) is necessary as the distribution on constraints in the sub-game (H o G o 
H)sxw (as defined in (3.1)) is essentially ys (as yr in this case is uniform). 

As for (3.3), let us assume that 


Taking T = S, we obtain that the distribution (defined in Equation (3.1)) induced by the game 
(H o G o H)sxS on the edges of G is given by 


TT 


hsjxf ^ ( |X| 

VI+ c 

where the last equality used the fact that ||l<s 11^ — || Ms 

|X| 




2 I II l|2 

+ U . 


E 


|X| V . .2 _ J_ 

c + lj ^ |X| 


1 + c 

IXI 

1 + c 


E 

xex 

E 

xex 


n{x) - 


2 C + 1 


|xp 


Ms(^) - 


vc+T 


|X| 


MsW 


vc+T 


> 


> 


> 


,_, r 

Vl -To) ;cgX 


MsW - 


Vc+T 


|X| 


Vl + c 
1 

Vl + c 


VTTT - i) - E 

' xex 


w(^) - |xi 


1 + c - 1 - £ . 


Thus, if the distribution on constraints is e-close to uniform, then the above lower bound forces 
C = 0(£). □ 


3.2 General (non-regular) extractors are insufficient 

Suppose H = {{W,X),Eh) is an arbitrary (<5,0(£))-extractor. Consider a possible scenario where 
there is a subset S C W with |S| > W| such that ys is of the form 

“ V' 1^1 - I'""' 1^1 - ly 

Notice that this is a legitimate distribution that may be obtained from a large subset S as |/^s “ 
is easily seen to be at most 2e. However, ifG = ((X, X),E) was d-regular with d = o(|X|), then 
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using (3.1), the probability mass on the edge (1,1) on the sub-game over S x S is 


= 



« 1 . 


In other words, if such a distribution jis be induced by the extractor, then the provers can 
achieve value close to 1 in the game (H o G o H)sxS by just labelling the edge (1,1) correctly. 
Thus, (H o G o H) is not even {3, 0.9)-robust. 

In Appendix A we show that we can adversarially construct a (^, 0(£))-extractor, although 
non-regular, that induces such a skew distribution. In Appendix B we also show that left-regular 
graphs of left-degree o(l/^£) are not fortifiers. 


4 Robustness from fortifiers 


In this section, we show that concatenating any two-prover game by fortifier(s) yields a robust 
game as claimed by Theorem 1.8. 

Lemma 4.1 (Distributions from large rectangles are close to uniform). Let ]j-g and jij be two proba¬ 
bility distributions such that 


Vs 


Vs 


< 


< Si and 
and 


(m 


Vt 


Vt 


< en 


< 


£2 

1^1 


(4.1) 

(4.2) 


Then for any bi-regular graph G = ((X, Y),E) that is a X^-expander, the distribution on edge {x, y) (where 
X e X and y G Y) given by (3.1) is (2ei +£\T- 2Ao • e 2 )-close to uniform. 

As described in Section 3, if Hi and H 2 are {3, £ 1 , £ 2 )-fortifiers, then for any set S and T of den¬ 
sity at least 3, the distribution on the constraints of (Hi o G o H 2 )sxT is given by (3.1). From the 
above lemma, it follows that the value of the game on any large rectangle can change only by the 
above bound on the statistical distance. By setting the parameters. Theorem 1.8 follows immedi¬ 
ately from Lemma 4.1. Further, Corollary 1.7 also follows from Lemma 4.1 and Lemma 2.8 as any 
graph is trivially a 1-expander. 


The rest of this section would be devoted to the proof of Lemma 4.1. For brevity, let us assume 
that |X| = n, I Y| = m and let d be the left-degree of G. We shall prove Lemma 4.1 by proving the 
following two claims. 


Claim 4.2. 


Claim 4.3. 


E 

(x,y)eG 


Vs{x)vt{v) 

L Vs{x)vT{y) 

{x,y)eG 


Vs{x)vT{y) 

dim 


< Aq • £2 


E 

{x,y)eG 


Vs{x)vT{y) 

dim 


1 

n ■ d 


< 2 £i -|- £^ -|- Aq • £2 
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Clearly, Lemma 4.1 follows from Claim 4.2 and Claim 4.3. 


Proof of Claim 4.2. If G denotes the normalized adjacency matrix of the graph G (that is, normal¬ 
ized so that Gux = uy), then observe that y)eGV-s{^)PT{y) = d ■ {Gys^Hr)- If we resolve }is 
and ^7 in the direction of the uniform distribution and the orthogonal component, we have 


{GVS,Vt) = (UY,Uy) + (G}lf,}lf'^ " ^ + (GFs^Ft) 


{Gys,VT) - - 


< Ao 


Fs 


Ft 


< 


Ao ■ £2 


m 


(using (4.2)) 


Therefore, 


E 

{x,y)eG 


Fs{x)FT{y) ys{x)FT{y) 


d {G}Is,Ft) 


d/m 


5. (ses) .. 


< 

{x,y)eG 
< Ao • £2- 


Proof of Claim 4.3. 


E 

{x,y)eG 


Fs{x)FT{y) 




d/m n ■ d 

Since fisix) = ^ + Fsi^) ^T{y) = ^ + pHy)' 

'm\ 


rm 

\~d 


E 

{x,y)eG 


Fs{x)FT{y) - 


n ■ m 


m\ 

l) 


E 

(x,y)eG 


Fsix)FT{y) - 
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n ■ m 


d) 


E 

{x,y)^G 


fs^A , f^T(y) 


m 


Fs{x)FHy) 


(Using triangle inequality) < E Fsi^) +-^ fHp) 

{x,y)eG {x,y)eG 

+ (7) E \Fs{x)FHy) 


(x,y)eG 


Fs 


+ 


Ft 


+ (7) E \Fs{x)FHy) 

{x,y)eG 


□ 


where the last equality uses the fact that G is a bi-regular graph. Define fs {x) = (x) | is a vector 

with the entrywise absolute values of fif, and similarly fj. Then, the RHS above equation reduces 
to 


Fs 


+ 


Ft 


, + (f) E 

(x,y)eG 


Fs 


-h 


Ft 


+ (7) • E fsix)fT{y) 


{x,y)eG 


Fs 


Ft 


+ m{Gfs,fT) 


(Using (4.1)) 


< 2 £i -I- m • (G/s,/r) • 


(4.3) 
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A simple bound for m • {Gfs,fT) would m ||Gf/g || \\^J\\ by Cauchy-Schwarz inequality. We can 
use the expansion of G again to estimate this better. Consider the decomposition/s = ai • ux + /^ 
and /j = a 2 • uy + fj. It follows that ai = |/s|^ and a .2 = |/t|i/ and hence ai,a 2 < £i by (4.1). 
Hence, 


m ■ {G/sJt) = ki-(X 2 + m - 


< £i 


(Using (4.2)) 


m ■ Aq 


'1 

< + m • Ao 

< £i + Aoe2- 


fs 

Fs 


/r 

Ft 



Combining this with (4.3), we get 


E 

(x,y)eG 


Fs{x)FT{y) 

d/m 


1 

n ■ d 


< 2^1 + + Ao£2- 
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A An explicit extractor that does not provide robustness 

Let H = {{W,X),Eh) be any (^5,£)-extractor. Let us assume that the extractor is left-regular with 
left-degree D, and let m = | W| and n = |X|. For any x E X and S C W, let ds (x) denote the degree 
of X in S. Let us fix one S C W such that |S| = <5| W|. 

We will transform the graph H so that the distribution induced by the set S looks like the 
counter-example described in Section 3.2 in the following two steps by altering the edges in the 
subgraph S x X: 

1. First change the degree into X from S to be exactly uniform. 

2. Next further change the degrees into X from S to be like the counterexample 

Both these operations can be achieved in a monotone fashion: for every x E X, the neighborhood 
of every vertex is either a superset, or a subset of its neighborhood before each operation. 

We will show that moving the edges this way does not perturb the indegree distribution from 
other large sets by too much, and the resulting graph is a (<5,0(e)) extractor as long as the number 
of edges we relocate is at most 0(ed ■ mD). This process will preserve the left-regularity of H but 
would not preserve bi-regularity. 


First let us move edges (monotonically) from S into X create the uniform distribution on X. 
When doing this, the degree of each vertex changes by A 5 (x) := \ds{x) — where ds(x) was 
the old degree. From the extractor property, we know that: 


E ^s{x) 

xex 


xex 


ds{x) 

Lds{x) 



< eS ■ mD. 


(A.1) 


Every vertex x E X now has degree dfvg- some vertex Xi G X, and relocate from every 
other X ^ x\ any set of e • edges to be incident on xi. Thus, if dg(x) refers to the new degrees, 
we have dg(xi) is (1 -|- en)d^yg where as d'g{x) is (1 — e)d®yg for every other x -h x\. 

The further change in degrees incurred on any x G X is ^'g{x) := |dg(x) — Since we this 
process only relocates 0{e ■ dayg|X|) edges, we have 


E 

xex 


L 

xex 


d's{x)-dl^g 


< 0{n-e- df^g) 


0{e3-mD). (A.2) 


Thus, the neighbourhood of any vertex x has changed additively by at most As(x) -|- Ag(x). 
Therefore, for any subset T C W of size at least d| W|, 


E 

d'Ax) - dl 

< 

E 

dr(x) - 

davg 

+ 

E \dTix) - dT{x)\ 

xex 


xex 




xex 


< 

e|T|D + 

E 

(As(^) 

+ A's(^)) 





xex 




< 

e|T|D + 

0(ed ■ mD) 

(using (A.l) and (A.2)) 


< 

0 (e 

■|T|D). 





Thus, the new graph after relocating edges is still an (d, 0(e))-extractor. This extractor, induces a 
distribution similar to the one described in Section 3.2 and hence carmot provide robustness. 
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B Lower bounds on degree of fortifiers 


In this section, we will show that an attempt to make a game (^, £)-robust by concatenating any 
left-regular graph with left degree D fail s if D < o(l/e^). 

Lemma B.l. Let H = ((W, X), Eh) be a left-regular bipartite graph with left-degree D = l/(c • e3) for 
some c > 0, and small enough constants e, S. Then, there exists a subset S C W with |S| > W| such that 
if p was the distribution on X induced by the uniform distribution on S then 


llp-uf > 


Q(c£) 

w 


Proof Lef davg = |kV|D/|X|. Nofe that at most |X|/2 vertices x satisfy deg(x) > Id^vg- Further, 
if there is a set S of |X| /4 vertices x that deg(x) < (0.5)davg, then if p is fhe disfribution on X in¬ 
duced by the uniform distribution on W, then |p — u|i > 1/4 which implies that 11p — u11 2 > 
by Cauchy-Schwarz. 


Otherwise, there exists X' C X such that |X'| = ce5^\X\ and for each x G X' we have 
(0.5)davg < deg(x) < Id^vg- Consider the set So of all neighbours of X'. If D < \/{cc5), we 
have I So I < 2c3^e ■ \ W\D = 2(5|W| which is a very small fraction of |W| when 3 is small enough. 
Consider an arbitrary set Si C W such that |Si| = 3m, with Si n So = 0. Let S 2 = So U Si- Let 
TTi, 712 be the probability distribution on X induced by Si, S 2 respectively. Note that IS 2 I < 3<5|W|. 

For every x G X', we know that 7ri(x) = 0 and 712 (x) = Q • Therefore, 


2 

711 — 71211 > n 


/c<52£|X|\ 

V <5^1X12 J 


n(c£) 

w 


Since || tti — 712 1| < || tti — u|| -|- || 712 — u||, we have that one of the sets Si or S 2 shows the validity 
of the lemma □ 


We thus immediately infer the following: 

Corollary B.2. For all small enough 3,e > 0, no left-regular graph H = ((W, X),Eh) left-degree 
D = o{1/£3) is an {3, *,e) -fortifier. 

Note that any (^, £,£)-fortifier is in particular an (^, £)-extractor, and hence we also have that 
D = n((l/£^) log(l/<5)) [RTOO]. We also point out that the construction of Lemma 2.8 has left- 
degree D = 0(l/£^^). The above essentially shows this construction is almost optimal. 


C Parallel repetition from fortification 

We present a mild generalization of Theorem 1.4 to general bi-regular games, following essentially 
the same strategy as in [Mosl4]. 

Lemma C.l. LetG = {{X,Y) ,E) be a {3 ,e)-robust general game that isbi-regular with23 {\Lx\\E,y\)^~^ < 
£. Then, 

val(G^) < val(G*^^^) ■ (val(G)-|-£) -|- £. 
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Proof. Consider any deterministic strategy for the provers. These are merely functions 

/i : ^ and ^ 

that assign labels to the k queries asked by the verifier. For every {k — 1)-tuple of queries v = 

{vi ,..with each Vi := G E, and an arbitrary tuple of {k — 1) pairs of labels a := 

e (^x X define the rectangle TZ^^a ■= x where 

Sv,a = {xk ■ fi {xi, X 2 ,..., Xk) assigns label (7, to x, for all i < fc — 1}, 

— {Vk • fiiyi/Vir ■ ■ ■ fVk) assigns label to y, for all i < k — 1} . 

Also we shall call a rectangle accepting if every coordinate (u",-, cf) oi a satisfies the constraint 
on Vi = {xi,yi) for all 1 < i < fc — 1. In words, an accepting rectangle TZ^^a is the set of all possible 
queries Vk for the last round such that the provers win on the first {k — 1) rounds with Xi,..Xk-i 
and yi,..., pk-i getting labels Ci,..., Ck-i and cf,..., ^ respectively. We shall call a rectangle 

TZf,^^ "large" if and have density at least S, and "small" otherwise. We shall partition the 
space of all possible queries (vi,.. .,Vk) into the following sets. Note that Vk belongs to a unique 
rectangle TZ-o,a- 

• Ao = {{vi, ...,Vk) : TZv,a is not accepting} 

• Ai = {{vi,... ,Vk) : TZv^a is accepting and "large") 

• A 2 = {{vi, ■ ■ ■ ,Vk) : 'TZv,d- is accepting and "small") 

Observe that |Ai| + IA 2 I < val(G*^^^) ■ \Ef because Ai U A 2 is the set of queries on which the 
provers succeed on the first {k — 1) rounds. 

Also, the projection of elements in set Ai to the kth coordinate, is essentially a union of large 
rectangles. By the (<5, e)-robustness of G, any strategy of the provers can succeed on each large 
rectangle with probability at most val(G) -|- e. Hence, the provers succeed on at most a (val(G) + 
e)-fraction of points in Ai. 

Furthermore, since G is regular, we get IGI 2 I is at most |E|*^ ^ A3\E\ ■ |Ex x < e |£|*^ by 

the choice of 3 and e.^ 

Hence, the total number of queries on which the provers can succeed is upper bounded by 
(val(G) -|- e) \Ai\ + \A 2 \. It therefore follows that they succeed on at most a val(G^^^)(val(G) -|- 
e) -h £ fraction of queries. □ 

Unfolding the recursion from the above lemma, we get the following generalization of Theorem 1.4. 
Corollary C.2. Let G = ((X,Y),E) tea {3, £)-robust general game with 23 {\T,x\\E.y\)^^^ < £• Then, 

val(G*^) < (val(G) + e)^ + k ■ e. 

As mentioned earlier, this is not useful when say |Ex| = exp(0(l/^i)), which is unfortunately 
the case when an arbitrary game is made robust by concatenating with a fortifier. 

‘^In the case of projection games, the set of a that are accepting pairs for v can be indexed with instead of 
(Ex X and that gets the better parameters for projection games as in Theorem 1.4. 
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D Making the graph bi-regular 

In this section, we shall show that a general game on a graph can be converted to a slightly larger 
game on a bi-regular graph with almost the same value. 

Lemma D.l. Given a two-prover game G any graph ((X, Y), E). For every e > 0, there is a polynomial 
time algorithm to construct a game G' with size(G') = size(G) ■ O ((|E,x| + |^y|)/£)^ such that G' is 
on a bi-regular graph and val(G') < val(G) + e. 

The rest of this section would be a proof of this. Suppose we have a graph G = ((X,Y),£) that 
is possibly non-regular. We shall make some transformations on the graph to make it bi-regular 
such that it does not affect the value of the game by much. This is along the same lines as the 
technique used by Dinur and Harsha [DH13]. We shall need the following well-known Expander 
Mixing Lemma. 

Lemma D.2 (Expander Mixing Lemma). Let H = {{P, Q), E) be a k-expander with |P| = |Q|. Then, 
for every subsets A Q P and B Q, 

|£(AB)| _ M H < A 

|E| \P\ ■ IQI - 

A proof of the above lemma may be found in any text that studies expanders graphs (for 
example, [AS92, Chapter 5]). 

We shall make the graph bi-regular in two steps. We shall first make a transformation that 
makes it regular on the right side, and then repeat the same process on the left. But first, we 
would need to ensure that the degree on the Y side is large enough for the transformation to 
work. This is just done by creating d copies of every edge with the same constraint. The graph 
therefore becomes a multi-graph but the value remains the same.^ 

Thus, from now on, we assume that we are given a game G= ((X, Y),E), with the minimum 
degree being "large enough", that we want to make biregular. The transformation of G to make it 
regular on right side is as follows (Figure 3): 

For every vertex y G X with degree dy, we shall have a set Gy of dy vertices. Between the ver¬ 
tices Gy and the neighbourhood of y (in G), we shall add a A-expander of degree d. The constraint 
on any edge between x E N{y) and a vertex in Gy would be the same as Let us denote this 

game by Ga. 

Lemma D.3. val(GA) < val(G) -|- A |Xy|. 

Proof. Consider any labelling La of Ga. From this, let L be the natural randomized labelling for G 
such that L(x) = Lx{x) for every x G X, and L{y) = Lx{yi) be where y, is a random element of 
Gy. For every y G Y, let 5y be the expected fraction of edges incident on y that are satisfied by this 
assignment. 

5y= Yj Li'[L(y) =o]-Pr [(L(x),cr) satisfies 

By the definition of val(G), we know that dySy < val(G) • |E|. 

yer 

®One could also do this by replicating every vertex d times and adding the edges between them. 
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Figure 3: Enforcing bi-regularity 

Subclaim D.4. For every y E Y, the fraction of edges between N{y) and Cy that are 
satisfied by Lx is at most {dy + A|Ey|) 

Before we prove this, let us see why this is sufficient to complete the proof of the lemma. The 
number of edges between Cy and N (y) is exactly d ■ dy where d is the degree of the expander. 
Therefore, the number of edges in Gx that are satisfied is 

Y, d ■ dy ■ {dy + X\i:y\) < d ■ dydy + O {dA \Zy\) ' dy 

yeY yeY yeY 

< (val(G) + A|Ey|)-|EA| 

as claimed by the lemma. Thus, it suffices to prove Subclaim D.4. 

Proof of Subclaim DA. The number of edges between Cy and N(y) is d ■ dy. Parti¬ 
tion the vertices of Cy into sets { Cy^a ■ tr G Ey} based on the label assigned by 
Lx. For every a G Ey, let Aa denote the set of vertices v G N(y) such that 
{Lx{x),(t) satisfies f'{x,y)- Hence, the set of edges that are satisfied by Lx is pre¬ 
cisely [JuE{Acr, Cy^a)- By Lemma D.2, 

1,1 \ d 

— I \ ' I I * Y A ■ d ■ dy 

fly 

^ ^ \-'^cr\ ■ |Cy,£ 7 | • — -|- A • |Ey| ■ d ■ dy 

ael-Y “y 

= (d • dy) Pr[L(y) = ^7] • Pr [{L{x),o-) satisfies 
aeZy 

A |^y| ■ d • dy 

= (dy -|- A • |Ey|) ■ {d ■ dy) 

as claimed, since the number of edges is d • dy. □ {Subclaim D.4) 

That hence finishes the proof of the Lemma. □ 

This operation ensures that the right-degree of the game Ga is d and the value changes by at 
most e/2 if A < (e/2 | Ey |) . By Lemma 2.4, we can choose explicit constructions of expanders with 
d = 0(1/ A^) = 0((|Ey| /e)^). The graph is now right-regular with degree d, and the degree of 
every x E X has increased by a factor of d. Repeating the same process for the other side makes 
both sides regular and the value changes by at most e. □(Lemma D.l) 


|E(^^.Cy,^)| 
Y \E{Aa,Cy,a)\ 

creT-Y 
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